The understanding capabilities of current state-of-the-art 3D models are limited by datasets with a small number of annotated data and a pre-defined set of categories. In its 2D counterpart, recent advances have shown that similar problems can be significantly alleviated by employing knowledge from other modalities, such as language. Inspired by this, leveraging multimodal information for 3D modality could be promising to improve 3D understanding under the restricted data regime, but this line of research is not well studied. Therefore, we introduce ULIP to learn a unified representation of image, text, and 3D point cloud by pre-training with object triplets from the three modalities. To overcome the shortage of training triplets, ULIP leverages a pre-trained vision-language model that has already learned a common visual and textual space by training with massive image-text pairs. Then, ULIP learns a 3D representation space aligned with the common image-text space, using a small number of automatically synthesized triplets. ULIP is agnostic to 3D backbone networks and can easily be integrated into any 3D architecture. Experiments show that ULIP effectively improves the performance of multiple recent 3D backbones by simply pre-training them on ShapeNet55 using our framework, achieving state-of-the-art performance in both standard 3D classification and zero-shot 3D classification on ModelNet40 and ScanObjectNN. ULIP also improves the performance of PointMLP by around 3% in 3D classification on ScanObjectNN, and outperforms PointCLIP by 28.8% on top-1 accuracy for zero-shot 3D classification on ModelNet40. Our code and pre-trained models will be released.

Unmanned combat air vehicle (UCAV) combat is a challenging scenario with continuous action space. In this paper, we propose a general hierarchical framework to resolve the within-vision-range (WVR) air-to-air combat problem under 6 dimensions of degree (6-DOF) dynamics. The core idea is to divide the whole decision process into two loops and use reinforcement learning (RL) to solve them separately. The outer loop takes into account the current combat situation and decides the expected macro behavior of the aircraft according to a combat strategy. Then the inner loop tracks the macro behavior with a flight controller by calculating the actual input signals for the aircraft. We design the Markov decision process for both the outer loop strategy and inner loop controller, and train them by proximal policy optimization (PPO) algorithm. For the inner loop controller, we design an effective reward function to accurately track various macro behavior. For the outer loop strategy, we further adopt a fictitious self-play mechanism to improve the combat performance by constantly combating against the historical strategies. Experiment results show that the inner loop controller can achieve better tracking performance than fine-tuned PID controller, and the outer loop strategy can perform complex maneuvers to get higher and higher winning rate, with the generation evolves.

Abstractive summarization is the process of generating a summary given a document as input. Although significant progress has been made, the factual inconsistency between the document and the generated summary still limits its practical applications. Previous work found that the probabilities assigned by the generation model reflect its preferences for the generated summary, including the preference for factual consistency, and the preference for the language or knowledge prior as well. To separate the preference for factual consistency, we propose an unsupervised framework named CoP by controlling the preference of the generation model with the help of prompt. More specifically, the framework performs an extra inference step in which a text prompt is introduced as an additional input. In this way, another preference is described by the generation probability of this extra inference process. The difference between the above two preferences, i.e. the difference between the probabilities, could be used as measurements for detecting factual inconsistencies. Interestingly, we found that with the properly designed prompt, our framework could evaluate specific preferences and serve as measurements for fine-grained categories of inconsistency, such as entity-related inconsistency, coreference-related inconsistency, etc. Moreover, our framework could also be extended to the supervised setting to learn better prompt from the labeled data as well. Experiments show that our framework achieves new SOTA results on three factual inconsistency detection tasks.

kNN-MT presents a new paradigm for domain adaptation by building an external datastore, which usually saves all target language token occurrences in the parallel corpus. As a result, the constructed datastore is usually large and possibly redundant. In this paper, we investigate the interpretability issue of this approach: what knowledge does the NMT model need? We propose the notion of local correctness (LAC) as a new angle, which describes the potential translation correctness for a single entry and for a given neighborhood. Empirical study shows that our investigation successfully finds the conditions where the NMT model could easily fail and need related knowledge. Experiments on six diverse target domains and two language-pairs show that pruning according to local correctness brings a light and more explainable memory for kNN-MT domain adaptation.

We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped into three themes: (1) visual navigation, (2) rearrangement, and (3) embodied vision-and-language. We discuss the dominant datasets within each theme, evaluation metrics for the challenges, and the performance of state-of-the-art models. We highlight commonalities between top approaches to the challenges and identify potential future directions for Embodied AI research.

3D肺部片段的重建在肺癌的外科治疗计划中起着重要作用，这有助于保存肺功能并有助于确保低复发率。但是，在深度学习时代，肺部段的自动重建仍未得到探索。在本文中，我们研究了是什么使肺部段自动重建。首先，我们在临床和几何上表达了肺部段的解剖学定义，并提出了遵守这些定义的评估指标。其次，我们提出了脉冲（隐式肺部段），这是一种旨在肺部段重建的深层隐式表面模型。通过脉冲自动重建肺部段的指标和视觉吸引力是准确的。与规范分割方法相比，冲动输出连续预测任意分辨率具有较高的训练效率和更少的参数。最后，我们尝试不同的网络输入，以分析肺部段重建任务中重要的事情。我们的代码可在https://github.com/m3dv/impulse上找到。

金融领域的数值推理 - 进行定量分析并总结了财务报告中的信息 - 可以大大提高业务效率并降低数十亿美元的成本。在这里，我们提出了一个数值推理问答系统，以回答财务文本和表数据源之间的数值推理问题，该问题由回收器模块，发电机模块和集合模块组成。具体而言，除了检索整个行数据外，我们还创新设计了一个细胞回收器，该池检索器可以检索金单元，以避免将同一行中的无关和相似的单元带到发电机模块的输入中。在发电机模块中，我们利用多个发电机来生产程序，这是回答问题的操作步骤。最后，在整体模块中，我们集成了多个程序，以选择最佳程序作为系统的输出。在FinQA竞争中的最终私人测试集中，我们的系统获得了69.79的执行精度。

知识图（KG）嵌入旨在学习连续矢量空间中kg的实体和关系的潜在表示。一个经验观察是，与相同关系相关的头部（尾巴）实体通常具有相似的语义属性 - 特别是它们通常属于同一类别 - 无论他们在kg中彼此之间有多远。也就是说，他们具有全球语义相似性。但是，许多现有方法基于本地信息得出了kg嵌入，这些信息无法有效地捕获实体之间的这种全球语义相似性。为了应对这一挑战，我们提出了一种新颖的方法，该方法引入了一组称为\ textit {\ textbf {关系原型实体}}的虚拟节点，以表示由相同关系连接的头和尾部实体的原型。通过强制实体的嵌入靠近其相关的原型的嵌入，我们的方法可以有效地鼓励实体的全球语义相似性（可以在kg中很远 - 通过相同的关系相连。实体一致性和KG完成任务的实验表明，我们的方法显着优于最近的最新方法。

聚类是一项基本的机器学习任务，在文献中已广泛研究。经典聚类方法遵循以下假设：数据通过各种表示的学习技术表示为矢量化形式的特征。随着数据变得越来越复杂和复杂，浅（传统）聚类方法无法再处理高维数据类型。随着深度学习的巨大成功，尤其是深度无监督的学习，在过去的十年中，已经提出了许多具有深层建筑的代表性学习技术。最近，已经提出了深层聚类的概念，即共同优化表示的学习和聚类，因此引起了社区的日益关注。深度学习在聚类中的巨大成功，最基本的机器学习任务之一以及该方向的最新进展的巨大成功所激发。 - 艺术方法。我们总结了深度聚类的基本组成部分，并通过设计深度表示学习和聚类之间的交互方式对现有方法进行了分类。此外，该调查还提供了流行的基准数据集，评估指标和开源实现，以清楚地说明各种实验设置。最后但并非最不重要的一点是，我们讨论了深度聚类的实际应用，并提出了应有的挑战性主题，应将进一步的研究作为未来的方向。

在这项工作中，我们探索了用于视觉接地的整洁而有效的基于变压器的框架。先前的方法通常解决了视觉接地的核心问题，即具有手动设计的机制，即多模式融合和推理。这样的启发式设计不仅复杂化，而且使模型容易过度拟合特定的数据分布。为了避免这种情况，我们首先提出了TransVG，该TransVG通过变压器建立了多模式的对应关系，并通过直接回归框坐标来定位引用区域。我们从经验上表明，复杂的融合模块可以用具有更高性能的变压器编码层的简单堆栈代替。但是，TransVG中的核心融合变压器是针对Uni-Modal编码器的独立性，因此应在有限的视觉接地数据上从头开始训练，这使得很难优化并导致次优性能。为此，我们进一步介绍了TransVG ++以进行两倍的改进。一方面，我们通过利用Vision Transformer（VIT）进行视觉功能编码来将框架升级到一个纯粹的基于变压器的框架。对于另一个人来说，我们设计了语言有条件的视觉变压器，以去除外部融合模块，并重用Uni-Modal vit进行中间层的视觉融合。我们对五个普遍数据集进行了广泛的实验，并报告一系列最先进的记录。